Disk drive storage defragmentation system

ABSTRACT

The present invention provides a disk drive storage defragmentation system, comprising providing a cache buffer system coupled to a host system, coupling a disk drive storage system to the cache buffer system, performing a defragmantation process on the disk drive storage system utilizing the cache buffer system and servicing a data access request by the host system from the cache buffer system.

TECHNICAL FIELD

The present invention relates generally to disk drive storage systems,and more particularly to a system for defragmentation of file systems ondisk drive storage systems.

BACKGROUND ART

Disk drive based storage systems are widely used for storing digitaldata of all kinds, such as computer data and multimedia data (voice,video and image). In storage systems, one or more disk drives areconfigured to provide storage of data content. The disk drives can beread from or written to. Data can be stored and modified on the diskdrives via the computer host bus controllers. Before data is stored ontostorage systems, it must be organized in logic units, called data files,such as video files, image files, database files, etc. Such organizationis typically handled by the operating systems of the computer systemthat hosts the data files.

In general disk storage system can not hold data without a file system,rather it holds physical arrays of unspecified data blocks. A filesystem is a data structure layer implemented as part of the operatingsystem. It defines how the computer interfaces with the attached diskstorage, be it directly attached or attached through a network interfacecable. The file system defines how the data is organized and located onthe disk drives, file ownership and quotas, date of creation and change,and any recovery information associated with the file. The file systemis the critical link between the logic data files and the physical diskdrive storage systems. It not only manages the data files but also mapsthe files to the disk drive storage system.

To write a file to the disk storage, a sufficient number of data sectorsmust first be allocated. This operation is performed by the file systemand lower level device driver programs. Preferably all of the data of afile is stored on a single track and in consecutive data sectors. Inthis way, data is accessed continuously without moving the diskactuator.

However as host computers typically add and delete data files, causingallocated data blocks be freed and then allocated for a different file,possibly of different file size, the disk actuator must be moved fromtrack to track. The times required for the movement of the disk actuatorbetween two tracks on the disk drive platter, in order to gain access todata on different tracks, is called random seek latency. During thislatency, no data access is possible, therefore, it is preferred to havea seek latency as short as possible and preferred to have the number ofseek activities as low as possible.

As the file system stores and updates files on the disk drive, availablecontiguous data blocks become difficult to locate. If a file is largerthan the available contiguous data blocks, the file is fragmented andstored wherever the available blocks can accommodate it. In order toread or write the entire file, the disk drive actuator is moved to thetrack containing the first file segment. When the data for the firstfile segment is completed, a seek operation is performed to get to thetrack holding the second file segment. The seek operation can takeanywhere from 1 millisecond (mS) to 20 mS. A standard 3.5″ disk driveplatter rotating at 7200 RPM will make one revolution in 139microseconds (μS). During the shortest seek, the disk drive platter willcomplete 7.2 revolutions. The longest seek will allow the disk tocomplete 143 revolutions just to get to the track that contains therequested data. The average seek time on a 3.5″ disk drive isapproximately 8 mS, meaning the disk drive platter will complete 57revolutions, on average, every time the disk drive actuator is moved.

An example of a file fragmented into five segments, would require a seekoperation between each block. In order to retrieve the data for a readof this file, a total of 228 revolutions of the disk drive platter arewasted with no data retrieved during the seek operations. This candevastate the performance of the disk drive storage system. Thefragmentation of the files on the disk drive will impact the performanceof the host computer as processes await the requested file.

As the host computer repeatedly performs the above file operations overlong periods of time, and as the file system starts to get full, i.e.,most of the data sectors on the disk drive platters will be allocated tostored files, allocating new data blocks becomes increasingly difficult.Specifically, a file system may need to allocate data sectors inphysically scattered locations in order to store the data from a singlefile. Since the host computer accesses a file as a single data entity,the file system must gather all of the data blocks associated with aspecific file when such a file access request is issued by the hostcomputer. This requires that the disk drive actuator move to read datasectors from different locations on the disk drive platter, resulting inrandom seek latencies. The problem of a single data file being stored inscattered locations is called file system fragmentation. A fragmentedfile system causes an excessive amount of random seeks to access filespace allocated for write and read.

For real-time applications, the storage I/O access throughputrequirements, i.e., the transfer rate between the disk drive storagedevice and other devices connected to it, are usually high. The I/Othroughput requirements are also getting higher as applications get moreadvanced. For example, a high-definition television signal, when inuncompressed format, can require as high as 186 megabytes per second(MBps) access throughput. For digital film production, the storage I/Othroughput can be as high as 1.2 gigabyte per second (GBps). Moreimportantly, the data transfer to the storage device must be performedisochronously, i.e., with timing constraints. This poses a substantialchallenge to the disk drive technology.

Disk drives can deliver very high data throughput as long as randomseeks are reduced to a minimum. With fragmented file systems, asubstantial amount of random seeking and latencies may be introduced dueto file fragmentation. This can significantly reduce the datathroughput, often to a level below what is required to support real-timeapplications such as high-definition video and digital film production.In general, by storing all of the data blocks associated with a givenfile consecutively on a disk platter, fragmentation could be reduced oreliminated. This goal can not be achieved without taking up additionalhost computer resources. Specifically, as the file system gets full, oras the file system gets an increasing amount of file write and deleteoperations, the fragmentation will reach a level that the host computertypically will initiate a processing task called defragmentation.

In a defragmentation process, the data blocks are re-arranged on thedisk drives so that the data blocks of the same file can be accessedwith the fewest number of random seeks. The defragmentation program willmove the data blocks of a given file to be on the same data track or inthe same general region. However, the defragmentation process requiresextra processing and data transfers. The host computer must decide, viathe file system, which data is moved to which location to minimizeresulting fragmentation. The objective is to reduce or eliminate thefragmentation of files. Many of today's operation systems havedefragmentation programs, and some prefer to run the defragmentation inthe background at a lower priority than application file accesses.During this process, some free data blocks may be used as a temporaryhold place for fragmented data blocks. Sometimes, if the disk drivedevice is almost full with user data blocks, there may not be enoughavailable free data blocks to hold the defragmented files. This meansthat certain fragmented data blocks may need to be copied multiple timeswithin the disk drive to move them to the desired and defragmentedlocations. As a result, the overall data access throughput is furtherreduced due to the defragmentation process, defeating the originalpurpose which is to improve the storage system performance.

Thus, a need still remains for a system to manage the amount offragmentation a file system is subjected to. In view of the throughputdemand generated by new applications, it is increasingly critical thatanswers be found to these problems. Solutions to these problems havebeen long sought but prior developments have not taught or suggested anysolutions and, thus, solutions to these problems have long eluded thoseskilled in the art.

DISCLOSURE OF THE INVENTION

The present invention provides a disk drive storage defragmentationsystem, comprising providing a cache buffer system coupled to a hostsystem, coupling a disk drive storage system to the cache buffer system,performing a defragmantation process on the disk drive storage systemutilizing the cache buffer system and servicing a data access request bythe host system from the cache buffer system.

Certain embodiments of the invention have other aspects in addition toor in place of those mentioned or obvious from the above. The aspectswill become apparent to those skilled in the art from a reading of thefollowing detailed description when taken with reference to theaccompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a disk drive storage defragmentation systemin an embodiment of the present invention;

FIG. 2 is a disk system block diagram of the cache buffer system, ofFIG. 1;

FIG. 3 is a disk block diagram of a file in a defragmantation process;

FIG. 4 is a flow chart of a defragmentation process, in an embodiment ofthe present invention;

FIG. 5 is an other flow chart depicting another defragmantation process,in an alternative embodiment of the present invention; and

FIG. 6 is a flow chart of a system for disk drive storagedefragmentation system in an embodiment of the present invention.

BEST MODE FOR CARRYING OUT THE INVENTION

In the following description, numerous specific details are given toprovide a thorough understanding of the invention. However, it will beapparent that the invention may be practiced without these specificdetails. In order to avoid obscuring the present invention, somewell-known circuits, system configurations, and process steps are notdisclosed in detail. Likewise, the drawings showing embodiments of theapparatus/device are semi-diagrammatic and not to scale and,particularly, some of the dimensions are for the clarity of presentationand are shown greatly exaggerated in the drawing FIGs. The same numbersare used in all the drawing FIGs. to relate to the same elements.

Referring now to FIG. 1, therein is shown a block diagram of a diskdrive storage defragmentation system 100 in an embodiment of the presentinvention. The block diagram of the disk drive storage defragmentationsystem 100 depicts a disk drive storage system 102, having multipleinstances of a storage disk 104, a cache buffer system 106 and a hostsystem 108. The host system 108 could be a single end user system or adata server that supplies user data to a vast network of end usersystems. The cache buffer system 106 is an intermediary device betweenthe host system 108 and the disk drive storage system 102.

The cache buffer system 106 utilizes a large random access memory whichis generally large enough to hold the entire one or more data files,such as up to a one terabyte memory or larger, to organize the data sentto the disk drive storage system 102. The cache buffer system 106 canactually manage the data written to an instance of the storage disk 104in order to minimize the fragmentation that occurs in normal operation.If there is no contiguous segment of free blocks large enough to holdthe file to be stored in contiguous fashion, the data can remain in thecache buffer system 106 until a large enough available storage segmentis identified rather than fragmenting the file into several dissociatedlocations on the storage disk 104. Sometimes in order to ensure that nodata loss occurs, the fragmented data file segments are stored on thedisk storage while all read requests are served directly out of thecached copy within the cache buffer. If the same data is requested froma read operation, the data can be transferred directly from the cachebuffer system 106.

The cache buffer system 106 stores a copy of the meta data, from thehost system 108, in order to maintain a valid map of the location of allfiles stored on the storage disk 104. The meta data is maintained by theoperating system and contains file level information such as thelocation, size, creation date, ownership, quota, etc. The meta data mustremain synchronized between the host system 108 and the cache buffersystem 106. Without the synchronization of the meta data, the filesystem would become corrupted and user data would be lost. The criticalnature of the meta data requires that a single version of the data bekept completely up to date. In order to facilitate this, the cachebuffer system 106 maintains the copy of the meta data file that can beaccessed by the host system 108 and will be modified whenever the hostsystem copy of the meta data file is modified.

Referring now to FIG. 2, therein is shown a system block diagram of thecache buffer system 106, of FIG. 1. The system block diagram of thecache buffer system 106 depicts a buffer 202, under the control of acache manager 204, a write controller 206, a read controller 208 and ameta data receiver 210. The buffer 202 is made up of an array of memorychips, such as DDR RAM. The buffer 202 may contain a large amount ofmemory, such as a terabyte or more. The buffer 202 is controlled by thecache manager 204. The meta data receiver 210 is the repository for thecopy of the file system meta data. This data can be either an integralpart of the file system, in which case, it is stored together with thefile data, or it can be stored on a separate meta data server, such asthe case with some shared file systems. The meta data is the filesystem's map to the physical location of all of the files stored in thedisk drive storage system 102, of FIG. 1.

The cache manager 204 determines what data will remain in the buffer 202and whether that data should be written to the storage disk 104, ofFIG. 1. If a read command is received from the host system 108, of FIG.1, the cache manager 204 will determine if the requested data is locatedin the buffer 202. If all of the data or the initial segments reside inthe buffer 202, the cache manager 204 will enable the read controller208 to start the transfer from the buffer 202 to the host system 108.The read controller 208 also reads the meta data from the meta datareceiver 210. If the file is fragmented, the read controller 208 maycommunicate to the cache manager 204 to save a copy of the data as well.This copy of the data can be written back to the storage disk 104, in anarea of contiguous free blocks at a later time and then delete thefragmented copy, thus eliminating the fragmentation of the file.

When the host system 108, issues a write command, the write controller206 activates the path to the cache manager 204 so the data can becopied to the buffer 202 first. While the data is being buffered, thefile system allocates the disk storage segments so that the file can bestored in the disk storage and create entries in the meta data fileallocation table. Keep in mind that at this time, the actual data fileis still in the cache buffer. The modified meta data is also copied intothe meta data receiver. After that the write controller 206 interrogatesthe file allocation table in the meta data receiver 210. If thecircuitry within the write controller 206 determines that the file isfragmented, it may elect to defer the write to the storage disk 104, andkeep the data in the buffer 202. Further accesses of this data by thehost system 108, are directed to the cache manager 204. The writecontroller 206 works with the cache manager 204 in order to identify anarea where the file can be written in contiguous data blocks on thestorage disk 104. Once such an area is identified, the cached data iscopied back to the storage disk 104, preferably at a time when otheractivity level is low. This deferred write capability has a significantimpact on reducing the fragmentation of the storage disk 104.

A way to deal with the risk of power failure is to allow the data writeback to the storage disk 104 even if the allocated disk space issegmented, resulting in fragmented file storage. The cached copy of thedata remains in the buffer 202, therefore, the file fragments on thestorage disk 104 will not be read. The cached data will be read, whilethe file fragments stored on the storage disk 104 function as backupcopy of the data file in the buffer 202.

The cache manager 204 takes an active part in the defragmentationprocess as well. If an instance of the storage disk 104, becomes toofragmented, the host system 108, can initiate a defragmentation process.In a two step operation, the cache manager 204 may instruct the readcontroller 208 to read the entire content of the storage disk 104, andstore it in the buffer 202. The data could be reordered in the buffer202 and the cache manager 204 may instruct the write controller 206 towrite the newly ordered data back to the storage disk 104. Or morepreferably the data may be in the buffer 202 in the same layout as itappears in the disk sectors, and be reordered as it's written back tothe storage disk. The random access nature of the buffer 202 enablesout-of-order read from the buffer 202 for in-order write to the storagedisk 104.

If a read request is received by the read controller 208 to access datathat is involved in the defragmentation process, the cache manager 204enables the transfer from the buffer 202. In this fashion, thedefragmentation process does not degrade the performance of the diskdrive storage system 102. Any transaction that is serviced from thebuffer 202 by the cache manager 204 will execute much faster thanretrieving the data from the storage disk 104. The buffer 202 can accessthe data approximately 1000 times faster than the storage disk 104. Aread request of a fragmented file on the storage disk 104, will bedetected by the circuitry of the read controller 208. The data will betransferred from the storage disk 104 to the buffer 202 and the hostsystem 108. The read controller 208 activates a status register withinthe cache manager 204 indicating the fragmented file should be writtenback to the storage disk 104 in contiguous free data blocks at a latertime.

If the buffer 202 does not have enough available space to copy theentire contents of the fragmented instance of the storage disk 104, thedefragmentation process proceeds with the cache manager 204 examiningthe meta data to determine the location of all of the file fragments.The strategy of the defragmentation process is to create largecontiguous free blocks on the storage disk 104. In order to facilitatethis goal, file fragments that reside between free contiguous blocks aremoved first. The file fragments are transferred to the buffer 202 by thecache manager 204. This creates a larger contiguous section of freeblocks. For data reliability purposes, the relocated data fragmentshould be stored back to the storage disk 104 before the originalfragment is deleted. If the buffer 202 is assumed to be reliable, thefragment can remain in the buffer 202 even after the original copy ofthe fragment has been deleted from the storage disk 104.

The criteria used to identify the target file fragment is related by thesize of the file fragment, β, and the size of the two neighboring freeblocks, F₁ and F₂. The fragment presenting the smallest ratio R of theoutput of equation 1 is the first candidate to move.R=β(F ₁ +F ₂)÷(F ₁ ×F ₂)  (1)Another way to put the above expression is:R=β(1/F ₁+1/F ₂)  (2)Essentially what this means is that smaller fragments get relocatedfirst, fragments with larger contiguous neighboring free blocks arerelocated first. Therefore, the above expression is only one of manypossible ways to make this selection. For example, just for the sake ofargument, another way to make the selection would be:R=β(1/F ₁ ²+1/F ₂ ²)  (3)This would make similar choices with somewhat different weight on beta,F1 and F2. In this example, more weight, or sensitivity, is assigned tothe values of F1 and F2 than to beta. So we need to point out that theselection criteria above are for illustration purposes only.

The preferred criteria will select the smallest fragments that willliberate the largest contiguous free block space. The criteria favorscases where both F₁ and F₂ are close in size and both are large. It alsofavors cases where the relative size of the free blocks, F₁ and F₂, arelarger than the file fragment, β. The goal of the criteria is togenerate as much free contiguous space as possible with the least amountof data moved. The circuitry within the cache manager 204 implementsthis algorithm in order to determine which file fragments should berelocated.

The file fragments identified by the criteria are moved into the buffer202 under control of the cache manager 204 and executed by the readcontroller 208. By selecting the smallest file fragments to relocateinto the buffer 202, the cache manager 204 can manage a higher number offragmented files. As the number of contiguous free blocks increases andthe fragmented files are assembled in the buffer 202, the cache manager204 starts to restore the files on the storage disk 104. The selectionand transfer process continues until all of the fragmented files havebeen restored in contiguous space on the storage disk 104. During thedefragmentation process any read or write operations to files held inthe buffer 202, are executed by the cache manager 204 without accessingthe storage disk 104.

The cache manager 204 updates the meta data in the meta data receiver210 as files are moved. If the host system 108, requests access to aspecific file, the meta data receiver 210 utilizes the updated meta datato set-up the transfer. If the data resides in the buffer 202 the cachemanager 204 will notify the meta data receiver 210 that the data accesscan commence to the buffer 202. The meta data receiver 210 then passesthe file location information to the write controller 206 or the readcontroller 208 depending on the type of access requested.

Referring now to FIG. 3, therein is shown a disk block diagram 300 of afile in a defragmantation process. The disk block diagram 300 depictsthe resultant location of the once fragmented segments of the file. Thefile fragments have been restored in a contiguous format on the storagedisk 104, of FIG. 1. The file can be accessed in one revolution of thedisk without seek latency after the defragmantation process, where ittook 228 revolutions to access the file in the previous fragmentedstate. In the defragmented state, the first block 302 is adjacent to thesecond block 304, which is adjacent to the third block 306, which isadjacent to the fourth block 308 which is adjacent to the fifth block310.

Referring now to FIG. 4, therein is shown a flowchart 400 of thedefragmentation process, in an embodiment of the present invention. Theflowchart 400 depicts the logical process flow used by the cache manager204, of FIG. 2. A first decision block 402 tests the meta data todetermine whether all the file fragments on the storage disk 104, ofFIG. 1, will fit in the buffer 202, of FIG. 2. If all the file fragmentswill fit in the buffer 202, a copy block 404 performs the copy of allthe file fragments into the buffer 202. A restore block 406 reorders thefragmented files and writes them back to the storage disk 104, incontiguous data blocks and updates the meta data. At the completion ofthe restore block 406, a first transition 430 takes the process to anend block 432 to terminate the defragmentation process.

If the first decision block 402 determines that all of the filefragments will not fit in the buffer 202, a second transition 408 takesthe process to a find free block 410. The find free block 410 examinesthe meta data to locate a largest contiguous free block region on thestorage disk 104. Having found the largest contiguous free block, theprocess moves to an adjacent fragments block 412 locates file fragmentsthat are adjacent to the previously identified free block region. Asecond decision block 414 tests to determine whether the buffer 202, canhold all of the adjacent file fragments. If the buffer 202, can not holdthe identified adjacent file fragments, a move file fragments block 416copies the file fragments to another area of free blocks on the storagedisk 104, and updates the meta data. At the completion of the move filefragments block 416, the process enters the second transition 408 foranother attempt.

If the buffer 202, can hold all of the adjacent file fragments, a moveto buffer block 418 reads the data from the adjacent file fragments intothe buffer 202 and frees the data blocks that held the adjacent filefragments and updates the meta data, thus increasing the size of thelargest contiguous free block region. The next process step is a findsmallest block 420 searches the meta data for the smallest fragmentedfile on the storage disk 104. A third decision block 422 tests todetermine whether the smallest fragmented file will fit into the largestcontiguous free block region. If the smallest fragmented file will fitinto the largest contiguous free block region, a copy smallest block 424moves the smallest fragmented file from its diverse locations into thelargest contiguous free block region as a complete and contiguous file,and updates the meta data. At the completion of the copy smallest block424, a fourth decision block 426 examines the meta data to determine ifthere are any more fragmented files on the storage disk 104. If thereare not any more fragmented files on the storage disk 104, the firsttransition 430 takes the process to the end block 432 to terminate thedefragmentation process. If the fourth decision block 426 determinesthat there are additional fragmented files, the process enters thesecond transition 408 for another attempt.

If the third decision block 422 determines that the smallest fragmentedfile will not fit into the largest contiguous free block region, a holdfragments block 428 updates the meta data and the process enters thesecond transition 408 for another attempt. By iterating through thisprocess all of the fragmented files will moved into contiguous files onthe disk. If at any time during the defragmentation process the hostsystem 108, requests access to data on the instance of the storage disk104, that is being processed, the data will be accessed from the buffer202, or from the storage disk 104. If the access creates additionalfragmented files segments, they will be defragmented in a followingiteration of the defragmentation process. In some configurations, thisdefragmentation process can run continuously in the background,maintaining peak performance of the disk drive storage system 102,without host system 108 intervention.

Referring now to FIG. 5, therein is shown another flowchart 500depicting another defragmentation process, in an alternative embodimentof the present invention. The other flowchart 500 depicts the logicalprocess flow used by the cache manager 204, of FIG. 2. Similarly thefirst decision block 402 tests the meta data to determine whether allthe file fragments on the storage disk 104, of FIG. 1, will fit in thebuffer 202, of FIG. 2. If all the file fragments will fit in the buffer202, the copy block 404 performs the copy of all the file fragments intothe buffer 202. The restore block 406 reorders the fragmented files andwrites them back to the storage disk 104, of FIG. 1, in contiguous datablocks and updates the meta data. At the completion of the restore block406, an alternative end block 518 terminates the defragmentationprocess.

If the first decision block 402 determines that all of the filefragments will not fit in the buffer 202, the second transition 408takes the process to the find free block 410. The find free block 410examines the meta data to locate a largest contiguous free block regionon the storage disk 104. Having found the largest contiguous free block,the process moves to the adjacent fragments block 412 locates filefragments that are adjacent to the previously identified free blockregion. The second decision block 414 tests to determine whether thebuffer 202, can hold all of the adjacent file fragments. If the buffer202, can not hold the identified adjacent file fragments, the move filefragments block 416 copies the file fragments to another area of freeblocks on the storage disk 104, and updates the meta data. At thecompletion of the move file fragments block 416, the process enters thesecond transition 408 for another attempt.

If the buffer 202, can hold all of the adjacent file fragments, the moveto buffer block 418 reads the data from the adjacent file fragments intothe buffer 202 and frees the data blocks that held the adjacent filefragments and updates the meta data, thus increasing the size of thelargest contiguous free block region. A redirect block 502 modifies themeta data so that the data blocks held in the buffer 202, are treated aspart of the file system, with write and read permissions. In this stateany transaction with the effected files will be made to the buffer 202.An activity check 504 checks to see if there is sufficient disk I/Othroughput to defragment the file. If there is not sufficient disk I/Othroughput then the buffer 202, remains the primary source of the dataand the system continues to check for available disk I/O throughput. Ifthere is available disk I/O throughput available the process moves to adisable write buffer block 506. The disable write buffer block 506disables the write permission on the file held in the buffer 202. Thefile is still accessible for read, but is not available formodification.

A disable cache block 508 turns off the caching algorithm so that thefile will remain in tact during the relocation to the storage disk 104.A restore file block 510 moves the file from the buffer 202, to an areaof the storage disk 104, that has sufficient free data blocks to holdthe file in contiguous blocks. An enable write buffer block 512 enablesthe write permission on the file in the buffer 202. An enable cacheblock 514 enables the caching algorithm and a free storage block 516deletes the old data blocks, from the files original fragmented blocks,freeing them for further use. Then the process returns to the firstdecision block 402 in order to find the next file to defragment.

Referring now to FIG. 6, therein is shown a flow chart of a system 600for disk drive storage defragmentation system 100 in an embodiment ofthe present invention. The system 600 includes providing a cache buffersystem coupled to a host system in a block 602; coupling a disk drivestorage system to the cache buffer system in a block 604; performing adefragmantation process on the disk drive storage system utilizing thecache buffer system in a block 606; and servicing a data access requestby the host system from the cache buffer system in a block 608.

In greater detail, a method to provide a disk drive storagedefragmentation system 100, in an embodiment of the present invention,is performed as follows:

-   -   1. Providing a cache buffer system coupled to a host system,        wherein the cache buffer system is an extension of the file        system structure. (FIG. 1)    -   2. Coupling a disk drive storage system to the cache buffer        system, further comprises reducing fragmentation of a storage        disk. (FIG. 1)    -   3. Performing a defragmantation process on the disk drive        storage system utilizing the cache buffer system in a background        operation. (FIG. 1)    -   4. Servicing a data access request by the host system from the        cache buffer system. (FIG. 1)

It has been discovered that disk drive storage defragmentation systemcan enhance the performance of the disk drive storage system in normaloperation. The features that reduce fragmentation allow the access offiles to be performed in the fastest possible time. It has beendiscovered that the present invention thus has numerous aspects. Anaspect is that the present invention a background defragmentationprocess can run without additional overhead for the host system. Anotheraspect is that the host system can access all files during thedefragmentation process without reduced performance. These and othervaluable aspects of the present invention consequently further the stateof the technology to at least the next level.

Thus, it has been discovered that the disk drive storage defragmentationsystem method and apparatus of the present invention furnish importantand heretofore unknown and unavailable solutions, capabilities, andfunctional aspects for preserving disk drive system performance. Theresulting processes and configurations are straightforward,cost-effective, uncomplicated, highly versatile and effective, can beimplemented by adapting known technologies, and are thus readily suitedfor efficiently and economically manufacturing devices that are fullycompatible with conventional manufacturing processes and technologies.

While the invention has been described in conjunction with a specificbest mode, it is to be understood that many alternatives, modifications,and variations will be apparent to those skilled in the art in light ofthe aforegoing description. Accordingly, it is intended to embrace allsuch alternatives, modifications, and variations which fall within thescope of the included claims. All matters hithertofore set forth hereinor shown in the accompanying drawings are to be interpreted in anillustrative and non-limiting sense.

For cases when the data must be protected, while the fragments arecopied into the cache buffer and being relocated, the old fragments cannot be deleted before the fragments are reassembled on a differentcontiguous location on the disk storage. If a file is considered to behighly fragmented but the storage can not accept the additional I/O loadof defragmentation, the file fragments remain on the disk storage whilea copy of it is being accessed from the cache buffer. This is clearlyacceptable for read purposes and should be considered as an importantaspect of the current invention since it improves read performanceduring defragmentation process.

For write request, the first time only the buffered copy gets written tofirst, creating an incoherent cached copy relative to the copy on disk.In this case, if data protection is required for content in buffer, thecached data copy may be copied back to the disk storage before the writerequest is confirmed successful.

1. A disk drive storage defragmentation system, comprising: providing acache buffer system coupled to a host system; coupling a disk drivestorage system to the cache buffer system; performing a defragmantationprocess on the disk drive storage system utilizing the cache buffersystem; and servicing a data access request by the host system from thecache buffer system.
 2. The system as claimed in claim 1, furthercomprising utilizing meta data by the cache manager to perform thedefragmentation process to minimize the data movement to and from thedisk drive storage system.
 3. The system as claimed in claim 1, furthercomprising: providing a cache manager in the cache buffer system;providing a write controller coupled to the cache manager; providing aread controller coupled to the cache manager; coupling a buffer to thecache manager; and providing a meta data receiver coupled to the cachemanager, to update physical location information of all files in thedisk drive storage system.
 4. The system as claimed in claim 1, furthercomprising pausing of the file defragmentation process if a data accessrequest arrives from the host system.
 5. The system as claimed in claim1, wherein utilizing the cache buffer system during the defragmentationprocess comprises servicing data access requests by the host system fromthe cache buffer system.
 6. A disk drive storage defragmentation system,comprising: providing a cache buffer system coupled to a host system,wherein the cache buffer system is an extension of the file systemstructure; coupling a disk drive storage system to the cache buffersystem; performing a defragmantation process on the disk drive storagesystem utilizing the cache buffer system in a background operation; andservicing a data access request by the host system from the cache buffersystem.
 7. The system as claimed in claim 6, further comprisingutilizing meta data by the cache manager to perform the defragmentationprocess.
 8. The system as claimed in claim 6, further comprising:providing a cache manager in the cache buffer system to manage thefragmentation of the storage disk; providing a write controller, coupledto the cache manager, to defer a write of a fragmented file; providing aread controller, coupled to the cache manager, to flag a fragmented filefor writeback; coupling a buffer to the cache manager, wherein thebuffer comprises an array of RAM to accommodate entirely the smallestmedia file stored in the disk drive storage system; and providing a metadata receiver to update physical location information on all files inthe disk drive storage system.
 9. The system as claimed in claim 6,further comprising pausing of the file defragmentation process if a dataaccess request arrives from the host system.
 10. The system as claimedin claim 6 wherein utilizing the cache buffer system during thedefragmentation process, comprises servicing data access requests by thehost system.
 11. A disk drive storage defragmentation system,comprising: a host system; a disk drive storage system; a cache buffersystem coupled between the host system and the disk drive storagesystem, wherein the cache buffer system defragments the disk drivestorage system and services data access request by the host system. 12.The system as claimed in claim 11, further comprising meta data utilizedby the cache manager to perform the defragmentation process to minimizethe data movement to and from the disk drive storage system.
 13. Thesystem as claimed in claim 11, further comprising: a cache manager inthe cache buffer system; a write controller coupled to the cachemanager; a read controller coupled to the cache manager; a buffercoupled to the cache manager, wherein the buffer accommodates thesmallest media file stored in the disk drive storage system; and a metadata receiver to update physical location information on all files inthe disk drive storage system.
 14. The system as claimed in claim 11,further comprises a data access request from the host system, whereinthe data access request pauses the file defragmentation process.
 15. Thesystem as claimed in claim 11, wherein data access requests by the hostsystem during the defragmentation process are serviced by the cachebuffer system.
 16. The system as claimed in claim 11, further comprisingan extension of the file system structure utilizing the cache buffersystem.
 17. The system as claimed in claim 16, further comprising metadata utilized by the cache manager to perform the defragmentationprocess and minimize the data movement to and from the disk drivestorage system.
 18. The claim as claimed in claim 16, furthercomprising: a cache manager in the cache buffer system to manage thefragmentation of the storage disk; a write controller, coupled betweenthe cache manager and a meta data receiver, controls write operationscomprising write data operations and deferring a write operation of afragmented file; a read controller, coupled between the cache managerand the meta data receiver, controls read operations comprising readdata operations and flagging a fragmented file for writeback; a buffer,coupled to the cache manager, wherein the buffer comprises an array ofRAM to accommodate entirely the smallest media file stored in the diskdrive storage system; and a meta data receiver, coupled between thecache manager, the write controller and the read controller, to updatephysical location information of all files in the disk drive storagesystem.
 19. The system as claimed in claim 16, further comprises a dataaccess request from the host system, wherein the data access requestpauses the file defragmentation process.
 20. The system as claimed inclaim 16, wherein the cache buffer system services data access requestsby the host system during the defragmentation process.